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е What is assessment? 


е What are recent trends in assessment 
in Orthopaedics? 


Why OSCE? 

















е History of OSCE in the local 
Orthopaedic department? 


What is the present status of OSCE in 
Orthopaedics. 


Recent trends in the other departments 
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eTo help the student learn EA. 
*This is called Formative ил. 


eTo assess the mission of 
the institution 
*This is called Values “Го test 


Assessment competence 
*This is called 
Reasons Summative 


for Assessment 
Assessment 





elo assess course outcomes 
*This is called Programme 
Assessment 
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What needs assessing? | | 


" Cognition or the processing of 
information 


" Knowledge 





Skills 
" What are other 3 things? ~, Attitude 
Cemmunication 
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Reliability 





"The assessment must measure the 


students' performance consistently 





Characteristics 
of 
Assessment 








"Every time the student is assessed a similar mark must be 


obtained 
"Strong students must score highly every time they take the 


examination and poor students must score low consistently 
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There аге 4 other characteristics of 
assessments... 





" What are they? 


" What do they measure? 
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Acceptability Reliability WS / 
«|t must be "The assessment must measure the 
acceptable to students' performance consistently 


patients, students Validi 
| alidi 
and examiners уалету 
| | 2 Assessment 
"This refers to privacy, must measure 


examination duration, "n 
Characteristics what you want 


patient fatigue, and 





marking time of to measure 
Assessment кк 

Educational І mpact Feasibility | 

"The assessment must drive "It must be feasible 


in terms of resources 


| "There must be sufficient 
| studens p he to have numbers of examiners with 
clinical skills, then these must be sufficient time and training to 


learning 





tested by an assessment which Will make the examination feasible 
motivate the student to acquire = 
these skills `` Orthopaedic Surgery 


Department of 





Егет Basic te Егет Simple te 


Abstract А 





Cemplex 








Knows how 





Knows 
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Selection o£ the assessment 
technique 
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Assessment of ‘knows’ 





е Knows 
- = straight factual recall of knowledge 


- Multiple choice question 


е А variety of formats 
- True/False 
- Single best answer 
- Extended matching questions 


е Only trivial knowledge tested 
- As active generation of knowledge is 
avoided 
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е Knows how 
- application of knowledge to problem 
solving and decision making 

- Essays & Oral Tests 

е Problems in standardizing 
е Inconsistency in marking 
- Despite this oral test are popular in ӘК 
е Ability to recall & synthesise best assessed 


е Short answer written formats 
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Ue vitm 
Assessment of ‘shows how’ and | 
‘does’ 


* ‘Mini СЕХ format’ 
- Observation of history taking 
- physical examination 
- Part management of case 














е OSCE 
- Globally well established. 
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е Addresses problem of: 
- Adequate sampling 


- Standardisation 
е Each case with same examiner 


- Improves reliability 
е Loss of validity 


- Cannot address complex skills 
requiring integrated judgement 








Department of 





- БА Orthopaedic Surgery 


осе test 








е OSCE is proving less objective 
than initially thought: 
- Scoring against checklist is not ideal 


- Global rating may be ‘ideal for 
judging performance". 








o Rater training is essential 
- Ensure consistency 
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° Standardised patient 
Vs 
Real patient 


° Simulated patient 


- extensive training 
е For reproducibility 
е Consistency of scenarios 


е OSCE # Real life performance 
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е Interaction 
with 
educators 
from 
Canada 


° PBL & 
OSCE 
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е Pathology Museum 
- МӘН Level 3 


е 75 students were examined at 
one time 
- 9 clinical cases 
е Foot drop 
е History taking 
— 9 stations 

е Projection 

е Pathology specimen 
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(с) 


These аге Xrays of а 5-year-old boy who complained o! pain in his left hip for about 6 months 


duration. Clinical examination revealed a shortened left lower limb and left hip movements were 
restricted. 


Described the features seen on the pelvic Xray. 
What is the diagnosis? 


What is the natural history of this condition? 


- This is a 50-year old female who presented to the clinic with progressive pain in the 
left forefoot 


a Describe the deformity shown in the photograph. 
b. Describe the possible causes of pain in this patient 


с Outline the management of this patient 
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b. 
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Describe the disability 


Diagnosis 


The appropriate investigation should be 





(© - This is a 50-year old female who presented to the clinic with progressive pain in the 
left forefoot 


а Describe the deformity shown in the photograph. 
b. Describe the possible causes of pain in this patient 


с Outline the management of this patient 


Department of 





Orthopaedic Surgery 


осе test 








ORTHOPAEDIC 


NATIONAL UNIVERSITY OF SINGAPORE 
DEPARTMENT OF SURGERY 


OSCE in ORTHOPAEDIC SURGERY 
This sixty five year old gentleman presented with pain and swelling in the arm after 
arm- wrestling with his neighbour. 
а What is the physical sign demonstrated ? 
в What is the cause of the swelling ? 


с What is your recommended treatment? 
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Reintroduction 


е Reintroduced OSCE test end of posting 
assessment as of July 2007. 











• | had 5 years experience of taking part in 
OSCE in Orthopaedics and for last 4 
years has been an examiner for the 
‘General Medical Council (GMC) United 
Kingdom’ for writing clinical scenarios 
for OSCE. 











• The idea is to use this as an end of 
posting assessment tool. 


Department of „5 m lationo 





2 Orthopaedic Surgery 


осе test 





Present Examination 





«Spine 
«Hip 
e Knee 
o Hand 
e Shoulder 
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e The first OSCE exam conducted on а 





Saturday morning over 3 hours 

- 80 students 

— four circuits of five stations each. 

- Time required for student to complete a cycle of 
five stations was 40 minutes (6 minutes per station 
& 2 minute between each patient which included 
transfer time and time for student to read the 
scenario). 


e Survey was then conducted on the 
‘assessors’ and ‘students’ 


е An ‘audit’ survey was then conducted to see 
the effect of changes introduced in the 
examination system 
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see Analysis of the Student's response 


STUDENT'S RESPOSES JULY 2007 | DEC 2007 


student Factors: 
Improves Psychomotor clinical skills 
Fear of Unknown 








Examiner Factors: 

Examiner not helpful 

Examiner Unaware of the examination pattern 
Interference by the examiner 


Scenario/ Case Related: 
Lack of time per station 
Inadequate time for studying the questions 


Examination System Suggestions: HE 


Well Organized 
Poorly Organized 
Pre-exam briefing poor 





Role Player Related: 
Role players did not play the part well 





Recommendations: 

Overall a good change 19 
Recommended to continue in the future 11 
Inclusion of discussion/ mini viva 5 
Real Patients would be better 3 
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Scenario related 

Scenario was realistic 

Task required was appropriate 
Sufficient Time allowed 
Improved examination skills 


Role player Related 
Simulated patient was consistent 


Played Role Realistically 


Recommendations: 

Should teach in a way wanting to examine 
Shorten the examination 

Include Discussions 
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OSCE well received. 


The clinical stations set were similar to the 
stations set for GMC's *Professional and 
Linguistic Assessment Board (PLAB) test, 
which has been standardized as a 
performance assessment test. 


From the ‘student’s’ & ‘examiners’ responses 
— it was good overall change 

- improves psychomotor skills of the students. 

Both ‘students’ and ‘examiners’ felt that 


— time was not adequate for the first round of 
examination was a mere indication of lack of the 
exposure to the assessment technique. 
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• There were clear indication by the 
students that they would be 
happy for this method of 
examination to be used as a 
method of assessment. 











e Examiners unaware of the 
examination technique and were 
interfering. 
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Examiner’s view point 


• There was an element of inertia and 
resistance to change that was evident 
from the responses of the examiners. 








е great reluctance by the examiners to 
follow the newly introduced 
examination technique and they 
wanted to continue with their 
established techniques and methods. 
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е Students will find it easy to adapt. 








е Examiners will find it difficult to 
adapt to the new method of 
assessment. 
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е Case based discussion (OSLER) 


- Objective structured Long 
examination of record 








е MCQ matching type. 
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е Presently we аге trialling OSCE as 
method of *end of posting assessment 


е Future we will have OSCE for end of 
posting assessment: 
- Increase the number of stations 









- Include 
е ‘role player station’ and 
е ‘interactive station’. 
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